<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
<head>
<title>Documentation</title>
<meta  http-equiv="Content-Type" content="text/html;charset=utf-8" />
<meta  name="generator" content="Org-mode" />
<style type="text/css">
 <!--/*--><![CDATA[/*><!--*/
  .title  { text-align: center; }
  .todo   { font-family: monospace; color: red; }
  .done   { color: green; }
  .tag    { background-color: #eee; font-family: monospace;
            padding: 2px; font-size: 80%; font-weight: normal; }
  .timestamp { color: #bebebe; }
  .timestamp-kwd { color: #5f9ea0; }
  .right  { margin-left: auto; margin-right: 0px;  text-align: right; }
  .left   { margin-left: 0px;  margin-right: auto; text-align: left; }
  .center { margin-left: auto; margin-right: auto; text-align: center; }
  .underline { text-decoration: underline; }
  #postamble p, #preamble p { font-size: 90%; margin: .2em; }
  p.verse { margin-left: 3%; }
  pre {
    border: 1px solid #ccc;
    box-shadow: 3px 3px 3px #eee;
    padding: 8pt;
    font-family: monospace;
    overflow: auto;
    margin: 1.2em;
  }
  pre.src {
    position: relative;
    overflow: visible;
    padding-top: 1.2em;
  }
  pre.src:before {
    display: none;
    position: absolute;
    background-color: white;
    top: -10px;
    right: 10px;
    padding: 3px;
    border: 1px solid black;
  }
  pre.src:hover:before { display: inline;}
  pre.src-sh:before    { content: 'sh'; }
  pre.src-bash:before  { content: 'sh'; }
  pre.src-emacs-lisp:before { content: 'Emacs Lisp'; }
  pre.src-R:before     { content: 'R'; }
  pre.src-perl:before  { content: 'Perl'; }
  pre.src-java:before  { content: 'Java'; }
  pre.src-sql:before   { content: 'SQL'; }

  table { border-collapse:collapse; }
  caption.t-above { caption-side: top; }
  caption.t-bottom { caption-side: bottom; }
  td, th { vertical-align:top;  }
  th.right  { text-align: center;  }
  th.left   { text-align: center;   }
  th.center { text-align: center; }
  td.right  { text-align: right;  }
  td.left   { text-align: left;   }
  td.center { text-align: center; }
  dt { font-weight: bold; }
  .footpara:nth-child(2) { display: inline; }
  .footpara { display: block; }
  .footdef  { margin-bottom: 1em; }
  .figure { padding: 1em; }
  .figure p { text-align: center; }
  .inlinetask {
    padding: 10px;
    border: 2px solid gray;
    margin: 10px;
    background: #ffffcc;
  }
  #org-div-home-and-up
   { text-align: right; font-size: 70%; white-space: nowrap; }
  textarea { overflow-x: auto; }
  .linenr { font-size: smaller }
  .code-highlighted { background-color: #ffff00; }
  .org-info-js_info-navigation { border-style: none; }
  #org-info-js_console-label
    { font-size: 10px; font-weight: bold; white-space: nowrap; }
  .org-info-js_search-highlight
    { background-color: #ffff00; color: #000000; font-weight: bold; }
  /*]]>*/-->
</style>
<link rel="shortcut icon" href="./favicon.ico" type="image/x-icon" />
<meta charset='utf-8'>
<meta http-equiv="X-UA-Compatible" content="chrome=1">
<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1">
<link href='https://fonts.googleapis.com/css?family=Architects+Daughter' rel='stylesheet' type='text/css'>
<link rel="stylesheet" type="text/css" href="stylesheets/stylesheet.css" media="screen" />
<link rel="stylesheet" type="text/css" href="stylesheets/pygment_trac.css" media="screen" />
<link rel="stylesheet" type="text/css" href="stylesheets/print.css" media="print" />
<header>
<div class="inner">
<h1>TimeData.jl</h1>
<h2>TimeData package for Julia language</h2>
<a href="https://github.com/JuliaFinMetriX/TimeData.jl" class="button"><small>View project on</small>GitHub</a>
</div>
</header>
<div id="content-wrapper">
<div class="inner clearfix">
<aside id="sidebar">
<a href="https://github.com/JuliaFinMetriX">
<img src="./logo.png" width="200" height="114">
</a>
<a href="https://github.com/JuliaFinMetriX/TimeData.jl/zipball/master" class="button">
<small>Download</small>
.zip file
</a>
<a href="https://github.com/JuliaFinMetriX/TimeData.jl/tarball/master" class="button">
<small>Download</small>
.tar.gz file
</a>
<p class="repo-owner"><a href="https://github.com/JuliaFinMetriX/TimeData.jl"></a> maintained by <a href="https://github.com/cgroll">cgroll</a>.</p>
</aside>
<section id="main-content">
<div>
<script type="text/javascript">
/*
@licstart  The following is the entire license notice for the
JavaScript code in this tag.

Copyright (C) 2012-2013 Free Software Foundation, Inc.

The JavaScript code in this tag is free software: you can
redistribute it and/or modify it under the terms of the GNU
General Public License (GNU GPL) as published by the Free Software
Foundation, either version 3 of the License, or (at your option)
any later version.  The code is distributed WITHOUT ANY WARRANTY;
without even the implied warranty of MERCHANTABILITY or FITNESS
FOR A PARTICULAR PURPOSE.  See the GNU GPL for more details.

As additional permission under GNU GPL version 3 section 7, you
may distribute non-source (e.g., minimized or compacted) forms of
that code without the copy of the GNU GPL normally required by
section 4, provided you include this license notice and a URL
through which recipients can access the Corresponding Source.


@licend  The above is the entire license notice
for the JavaScript code in this tag.
*/
<!--/*--><![CDATA[/*><!--*/
 function CodeHighlightOn(elem, id)
 {
   var target = document.getElementById(id);
   if(null != target) {
     elem.cacheClassElem = elem.className;
     elem.cacheClassTarget = target.className;
     target.className = "code-highlighted";
     elem.className   = "code-highlighted";
   }
 }
 function CodeHighlightOff(elem, id)
 {
   var target = document.getElementById(id);
   if(elem.cacheClassElem)
     elem.className = elem.cacheClassElem;
   if(elem.cacheClassTarget)
     target.className = elem.cacheClassTarget;
 }
/*]]>*///-->
</script>
<script type="text/javascript" src="http://orgmode.org/mathjax/MathJax.js"></script>
<script type="text/javascript">
<!--/*--><![CDATA[/*><!--*/
    MathJax.Hub.Config({
        // Only one of the two following lines, depending on user settings
        // First allows browser-native MathML display, second forces HTML/CSS
        //  config: ["MMLorHTML.js"], jax: ["input/TeX"],
            jax: ["input/TeX", "output/HTML-CSS"],
        extensions: ["tex2jax.js","TeX/AMSmath.js","TeX/AMSsymbols.js",
                     "TeX/noUndefined.js"],
        tex2jax: {
            inlineMath: [ ["\\(","\\)"] ],
            displayMath: [ ['$$','$$'], ["\\[","\\]"], ["\\begin{displaymath}","\\end{displaymath}"] ],
            skipTags: ["script","noscript","style","textarea","pre","code"],
            ignoreClass: "tex2jax_ignore",
            processEscapes: false,
            processEnvironments: true,
            preview: "TeX"
        },
        showProcessingMessages: true,
        displayAlign: "center",
        displayIndent: "2em",

        "HTML-CSS": {
             scale: 100,
             availableFonts: ["STIX","TeX"],
             preferredFont: "TeX",
             webFont: "TeX",
             imageFont: "TeX",
             showMathMenu: true,
        },
        MMLorHTML: {
             prefer: {
                 MSIE:    "MML",
                 Firefox: "MML",
                 Opera:   "HTML",
                 other:   "HTML"
             }
        }
    });
/*]]>*///-->
</script>
</head>
<body>
<div id="content">
<h1 class="title">Documentation</h1>
<div id="table-of-contents">
<h2>Table of Contents</h2>
<div id="text-table-of-contents">
<ul>
<li><a href="#sec-1">1. Getting started</a>
<ul>
<li><a href="#sec-1-1">1.1. Intuitive indexing</a></li>
<li><a href="#sec-1-2">1.2. Separate time information</a></li>
</ul>
</li>
<li><a href="#sec-2">2. TimeData types</a></li>
<li><a href="#sec-3">3. Constructors</a></li>
<li><a href="#sec-4">4. Indexing</a></li>
<li><a href="#sec-5">5. Read, write, io</a></li>
<li><a href="#sec-6">6. Conversion</a></li>
<li><a href="#sec-7">7. Data manipulation</a></li>
<li><a href="#sec-8">8. Functions and operators</a></li>
<li><a href="#sec-9">9. Iterators and map</a>
<ul>
<li><a href="#sec-9-1">9.1. Iterators</a></li>
<li><a href="#sec-9-2">9.2. Map</a></li>
<li><a href="#sec-9-3">9.3. Collapse</a></li>
<li><a href="#sec-9-4">9.4. Check condition</a></li>
</ul>
</li>
<li><a href="#sec-10">10. Plotting</a>
<ul>
<li><a href="#sec-10-1">10.1. Winston</a></li>
<li><a href="#sec-10-2">10.2. Gadfly</a></li>
</ul>
</li>
<li><a href="#sec-11">11. Additional functions</a></li>
<li><a href="#sec-12">12. Under the hood: implementation</a></li>
<li><a href="#sec-13">13. Current state</a></li>
<li><a href="#sec-14">14. Acknowledgement</a></li>
</ul>
</div>
</div>

<div id="outline-container-sec-1" class="outline-2">
<h2 id="sec-1"><span class="section-number-2">1</span> Getting started</h2>
<div class="outline-text-2" id="text-1">
<p>
The goal of the package TimeData is to provide <b>fast</b>, <b>robust</b> and
<b>convenient</b> representation of time series data. This shall be
achieved through introduction of several new composite types, matching
the characteristics of various types of time series data more closely.
</p>

<p>
Behavior of <code>TimeData</code> types in most cases borrows heavily from
<code>DataFrames</code>, as this package provides an excellent way of
representing quite general data. In particular, <code>DataFrames</code> already
allow for named columns, as well as missing observations. In many
situations, however, the time information of the data adds a separate
and unique dimension to the data. The different characteristics of
time data and observations then would not be accounted for
sufficiently, if time was only one further column amongst the other
observations.
</p>

<p>
The <code>TimeData</code> package provides users with the well-known experience
of <code>DataFrames</code> whenever possible. It only deviates where additional
convenience can be achieved through a more explicit separation between
time information and observations. In particular, the following
features allow convenient data handling.
</p>
</div>

<div id="outline-container-sec-1-1" class="outline-3">
<h3 id="sec-1-1"><span class="section-number-3">1.1</span> Intuitive indexing</h3>
<div class="outline-text-3" id="text-1-1">
<p>
Extending the fabulous ways of indexing already provided by
<code>DataFrames</code>, data can easily be accessed - amongst others - through
variable names and dates.
</p>

<div class="org-src-container">

<pre class="src src-julia">using TimeData
using Dates

fileName = joinpath(Pkg.dir("TimeData"), "data/logRet.csv")
tm = TimeData.readTimedata(fileName)[1:10, 1:4]
</pre>
</div>

<table border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides">


<colgroup>
<col  class="right" />

<col  class="right" />

<col  class="right" />

<col  class="right" />

<col  class="right" />
</colgroup>
<tbody>
<tr>
<td class="right">idx</td>
<td class="right">MMM</td>
<td class="right">ABT</td>
<td class="right">ACE</td>
<td class="right">ACT</td>
</tr>

<tr>
<td class="right">2012-01-03</td>
<td class="right">2.12505</td>
<td class="right">0.88718</td>
<td class="right">0.29744</td>
<td class="right">0.47946</td>
</tr>

<tr>
<td class="right">2012-01-04</td>
<td class="right">0.82264</td>
<td class="right">-0.38476</td>
<td class="right">-0.95495</td>
<td class="right">-0.52919</td>
</tr>

<tr>
<td class="right">2012-01-05</td>
<td class="right">-0.44787</td>
<td class="right">-0.23157</td>
<td class="right">0.28445</td>
<td class="right">2.74752</td>
</tr>

<tr>
<td class="right">2012-01-06</td>
<td class="right">-0.51253</td>
<td class="right">-0.93168</td>
<td class="right">0.23891</td>
<td class="right">1.94894</td>
</tr>

<tr>
<td class="right">2012-01-09</td>
<td class="right">0.58732</td>
<td class="right">0</td>
<td class="right">0.46128</td>
<td class="right">0.28436</td>
</tr>

<tr>
<td class="right">2012-01-10</td>
<td class="right">0.52193</td>
<td class="right">0.46693</td>
<td class="right">1.31261</td>
<td class="right">1.85986</td>
</tr>

<tr>
<td class="right">2012-01-11</td>
<td class="right">-0.63413</td>
<td class="right">-0.38895</td>
<td class="right">-1.52066</td>
<td class="right">-3.06604</td>
</tr>

<tr>
<td class="right">2012-01-12</td>
<td class="right">0.60934</td>
<td class="right">-0.46875</td>
<td class="right">0.50453</td>
<td class="right">-0.93039</td>
</tr>

<tr>
<td class="right">2012-01-13</td>
<td class="right">-0.80912</td>
<td class="right">0.50771</td>
<td class="right">-0.47478</td>
<td class="right">0.25752</td>
</tr>

<tr>
<td class="right">2012-01-17</td>
<td class="right">0.74711</td>
<td class="right">0.50515</td>
<td class="right">0.297</td>
<td class="right">-7.04176</td>
</tr>
</tbody>
</table>

<p>
Using a range of dates:
</p>
<div class="org-src-container">

<pre class="src src-julia">tm[Date(2012, 1, 4):Date(2012, 1, 10), 1:2]
</pre>
</div>

<table border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides">


<colgroup>
<col  class="right" />

<col  class="right" />

<col  class="right" />
</colgroup>
<tbody>
<tr>
<td class="right">idx</td>
<td class="right">MMM</td>
<td class="right">ABT</td>
</tr>

<tr>
<td class="right">2012-01-04</td>
<td class="right">0.82264</td>
<td class="right">-0.38476</td>
</tr>

<tr>
<td class="right">2012-01-05</td>
<td class="right">-0.44787</td>
<td class="right">-0.23157</td>
</tr>

<tr>
<td class="right">2012-01-06</td>
<td class="right">-0.51253</td>
<td class="right">-0.93168</td>
</tr>

<tr>
<td class="right">2012-01-09</td>
<td class="right">0.58732</td>
<td class="right">0</td>
</tr>

<tr>
<td class="right">2012-01-10</td>
<td class="right">0.52193</td>
<td class="right">0.46693</td>
</tr>
</tbody>
</table>

<p>
Using numeric indexing:
</p>
<div class="org-src-container">

<pre class="src src-julia">tm[3:8, 2:3]
</pre>
</div>

<table border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides">


<colgroup>
<col  class="right" />

<col  class="right" />

<col  class="right" />
</colgroup>
<tbody>
<tr>
<td class="right">idx</td>
<td class="right">ABT</td>
<td class="right">ACE</td>
</tr>

<tr>
<td class="right">2012-01-05</td>
<td class="right">-0.23157</td>
<td class="right">0.28445</td>
</tr>

<tr>
<td class="right">2012-01-06</td>
<td class="right">-0.93168</td>
<td class="right">0.23891</td>
</tr>

<tr>
<td class="right">2012-01-09</td>
<td class="right">0</td>
<td class="right">0.46128</td>
</tr>

<tr>
<td class="right">2012-01-10</td>
<td class="right">0.46693</td>
<td class="right">1.31261</td>
</tr>

<tr>
<td class="right">2012-01-11</td>
<td class="right">-0.38895</td>
<td class="right">-1.52066</td>
</tr>

<tr>
<td class="right">2012-01-12</td>
<td class="right">-0.46875</td>
<td class="right">0.50453</td>
</tr>
</tbody>
</table>

<p>
Using column names:
</p>
<div class="org-src-container">

<pre class="src src-julia">tm[3:8, [:ABT, :MMM]]
</pre>
</div>

<table border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides">


<colgroup>
<col  class="right" />

<col  class="right" />

<col  class="right" />
</colgroup>
<tbody>
<tr>
<td class="right">idx</td>
<td class="right">ABT</td>
<td class="right">MMM</td>
</tr>

<tr>
<td class="right">2012-01-05</td>
<td class="right">-0.23157</td>
<td class="right">-0.44787</td>
</tr>

<tr>
<td class="right">2012-01-06</td>
<td class="right">-0.93168</td>
<td class="right">-0.51253</td>
</tr>

<tr>
<td class="right">2012-01-09</td>
<td class="right">0</td>
<td class="right">0.58732</td>
</tr>

<tr>
<td class="right">2012-01-10</td>
<td class="right">0.46693</td>
<td class="right">0.52193</td>
</tr>

<tr>
<td class="right">2012-01-11</td>
<td class="right">-0.38895</td>
<td class="right">-0.63413</td>
</tr>

<tr>
<td class="right">2012-01-12</td>
<td class="right">-0.46875</td>
<td class="right">0.60934</td>
</tr>
</tbody>
</table>
</div>
</div>


<div id="outline-container-sec-1-2" class="outline-3">
<h3 id="sec-1-2"><span class="section-number-3">1.2</span> Separate time information</h3>
<div class="outline-text-3" id="text-1-2">
<p>
Through the hard-coded distinction between time data and observations,
code will become cleaner and less error-prone, as users can not mess
with time information as easily anymore. Functions that apply to
observations only, do not need to explicitly take into account the
time information anymore. For example, simply rescaling observations
by a factor of 2 becomes:
</p>

<div class="org-src-container">

<pre class="src src-julia">newTm = 2.*tm
</pre>
</div>

<table border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides">


<colgroup>
<col  class="right" />

<col  class="right" />

<col  class="right" />

<col  class="right" />

<col  class="right" />
</colgroup>
<tbody>
<tr>
<td class="right">idx</td>
<td class="right">MMM</td>
<td class="right">ABT</td>
<td class="right">ACE</td>
<td class="right">ACT</td>
</tr>

<tr>
<td class="right">2012-01-03</td>
<td class="right">4.2501</td>
<td class="right">1.77436</td>
<td class="right">0.59488</td>
<td class="right">0.95892</td>
</tr>

<tr>
<td class="right">2012-01-04</td>
<td class="right">1.64528</td>
<td class="right">-0.76952</td>
<td class="right">-1.9099</td>
<td class="right">-1.05838</td>
</tr>

<tr>
<td class="right">2012-01-05</td>
<td class="right">-0.89574</td>
<td class="right">-0.46314</td>
<td class="right">0.5689</td>
<td class="right">5.49504</td>
</tr>

<tr>
<td class="right">2012-01-06</td>
<td class="right">-1.02506</td>
<td class="right">-1.86336</td>
<td class="right">0.47782</td>
<td class="right">3.89788</td>
</tr>

<tr>
<td class="right">2012-01-09</td>
<td class="right">1.17464</td>
<td class="right">0</td>
<td class="right">0.92256</td>
<td class="right">0.56872</td>
</tr>

<tr>
<td class="right">2012-01-10</td>
<td class="right">1.04386</td>
<td class="right">0.93386</td>
<td class="right">2.62522</td>
<td class="right">3.71972</td>
</tr>

<tr>
<td class="right">2012-01-11</td>
<td class="right">-1.26826</td>
<td class="right">-0.7779</td>
<td class="right">-3.04132</td>
<td class="right">-6.13208</td>
</tr>

<tr>
<td class="right">2012-01-12</td>
<td class="right">1.21868</td>
<td class="right">-0.9375</td>
<td class="right">1.00906</td>
<td class="right">-1.86078</td>
</tr>

<tr>
<td class="right">2012-01-13</td>
<td class="right">-1.61824</td>
<td class="right">1.01542</td>
<td class="right">-0.94956</td>
<td class="right">0.51504</td>
</tr>

<tr>
<td class="right">2012-01-17</td>
<td class="right">1.49422</td>
<td class="right">1.0103</td>
<td class="right">0.594</td>
<td class="right">-14.08352</td>
</tr>
</tbody>
</table>

<p>
Functions that do not naturally extend to time data can simply be
overloaded and delegated to observations only.
</p>

<p>
Calculating mean values for each column:
</p>
<div class="org-src-container">

<pre class="src src-julia">mean(tm, 1)
</pre>
</div>

<table border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides">


<colgroup>
<col  class="right" />

<col  class="right" />

<col  class="right" />

<col  class="right" />
</colgroup>
<tbody>
<tr>
<td class="right">MMM</td>
<td class="right">ABT</td>
<td class="right">ACE</td>
<td class="right">ACT</td>
</tr>

<tr>
<td class="right">0.300974</td>
<td class="right">-0.0038740000000000107</td>
<td class="right">0.04458300000000001</td>
<td class="right">-0.398972</td>
</tr>
</tbody>
</table>

<p>
Calculating mean values per date:
</p>
<div class="org-src-container">

<pre class="src src-julia">rowmeans(tm)
</pre>
</div>

<table border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides">


<colgroup>
<col  class="right" />

<col  class="right" />
</colgroup>
<tbody>
<tr>
<td class="right">idx</td>
<td class="right">x1</td>
</tr>

<tr>
<td class="right">2012-01-03</td>
<td class="right">0.9472824999999999</td>
</tr>

<tr>
<td class="right">2012-01-04</td>
<td class="right">-0.261565</td>
</tr>

<tr>
<td class="right">2012-01-05</td>
<td class="right">0.5881325000000001</td>
</tr>

<tr>
<td class="right">2012-01-06</td>
<td class="right">0.18590999999999996</td>
</tr>

<tr>
<td class="right">2012-01-09</td>
<td class="right">0.33324</td>
</tr>

<tr>
<td class="right">2012-01-10</td>
<td class="right">1.0403325</td>
</tr>

<tr>
<td class="right">2012-01-11</td>
<td class="right">-1.402445</td>
</tr>

<tr>
<td class="right">2012-01-12</td>
<td class="right">-0.0713175</td>
</tr>

<tr>
<td class="right">2012-01-13</td>
<td class="right">-0.1296675</td>
</tr>

<tr>
<td class="right">2012-01-17</td>
<td class="right">-1.373125</td>
</tr>
</tbody>
</table>
</div>
</div>
</div>



<div id="outline-container-sec-2" class="outline-2">
<h2 id="sec-2"><span class="section-number-2">2</span> TimeData types</h2>
<div class="outline-text-2" id="text-2">
<p>
All types introduced are subtypes of abstract type <code>AbstractTimedata</code>
and have two fields, separating time information from observations:
</p>
<ul class="org-ul">
<li><b>vals:</b> a <code>DataFrame</code> 
</li>
<li><b>idx:</b> an <code>Array{T, 1}</code>, consisting of either type
<ul class="org-ul">
<li><code>Integer</code>,
</li>
<li><code>Date</code>, or
</li>
<li><code>DateTime</code> (from package <code>Dates</code>)
</li>
</ul>
</li>
</ul>

<p>
However, accessing these fields directly is considered poor style, as
one could circumvent any constraints on the individual fields this
way! Hence, only reference the fields directly if you really know what
you are doing.
</p>

<p>
Given these commonalities of all <code>TimeData</code> types, there exist several
distinct types that implement different constraints on the
observations. Sorted from most general to most specific case, the
following types are introduced:
</p>
<ul class="org-ul">
<li><code>Timedata</code>: no further restrictions on observations
</li>
<li><code>Timenum</code>: observations may consist of numeric values or missing
values only
</li>
<li><code>Timematr</code>: observations must be numeric values only
</li>
</ul>

<p>
Given that these constraints are fulfilled, one is able to define and
use more specific functions matching the data characteristics more
closely. For example, <code>Timematr</code> instances can directly make use of
fast and numerically optimized methods using <code>Array{Float64, 2}</code> under
the hood. Hence, it is important that these constraints are reliably
fulfilled. They are guaranteed as they are hard-coded into variables
at time of creation through the individual constructors. And, by now,
only a few methods exist that allow editing fields for data
manipulation, and they all are designed to maintain all restrictions
on the data.
</p>

<p>
The package generally tries to achieve high performance by delegating
functionality to the most specialized case. For example, methods for
instances of <code>Timematr</code> are delegated to <code>Array{Float64, 2}</code>, as they
are not allowed to entail <code>NAs</code> anyways.
</p>
</div>
</div>

<div id="outline-container-sec-3" class="outline-2">
<h2 id="sec-3"><span class="section-number-2">3</span> Constructors</h2>
<div class="outline-text-2" id="text-3">
<p>
For each type, variables can be created by directly handing over
observations as <code>DataFrame</code> and time information as <code>Array</code> to the
inner constructor.
</p>
<div class="org-src-container">

<pre class="src src-julia">vals = rand(4, 3);
dats = [Date(2013, 7, ii) for ii=1:4];
nams = [:A, :B, :C];
valsDf = composeDataFrame(vals, nams);

tm = Timematr(valsDf, dats)
</pre>
</div>

<table border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides">


<colgroup>
<col  class="right" />

<col  class="right" />

<col  class="right" />

<col  class="right" />
</colgroup>
<tbody>
<tr>
<td class="right">idx</td>
<td class="right">A</td>
<td class="right">B</td>
<td class="right">C</td>
</tr>

<tr>
<td class="right">2013-07-01</td>
<td class="right">0.3668076837657581</td>
<td class="right">0.6132177634585174</td>
<td class="right">0.7271614060006228</td>
</tr>

<tr>
<td class="right">2013-07-02</td>
<td class="right">0.8402870142445031</td>
<td class="right">0.8268085074961338</td>
<td class="right">0.7631093687543671</td>
</tr>

<tr>
<td class="right">2013-07-03</td>
<td class="right">0.6880695323767783</td>
<td class="right">0.03210864954311465</td>
<td class="right">0.9955387409975662</td>
</tr>

<tr>
<td class="right">2013-07-04</td>
<td class="right">0.8182674804881986</td>
<td class="right">0.7627315792913878</td>
<td class="right">0.966201684866371</td>
</tr>
</tbody>
</table>

<p>
Besides, there also exist several outer constructors for each type,
allowing more convenient creation. In particular, if observations do
not entail any <code>NAs</code>, there is no need to wrap them up into
<code>DataFrames</code> previously, but <code>TimeData</code> objects can simply be created
from <code>Arrays</code>. Also, there might be situations where variable names
and / or dates are missing. For these cases, there exist more
convenient outer constructors, too, which generally follow the
convention that dates never precede variable names as arguments.
</p>

<div class="org-src-container">

<pre class="src src-julia">td = Timedata(vals, nams, dats)
td = Timedata(vals, nams)
td = Timedata(vals, dats)
td = Timedata(vals)
</pre>
</div>

<table border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides">


<colgroup>
<col  class="right" />

<col  class="right" />

<col  class="right" />

<col  class="right" />
</colgroup>
<tbody>
<tr>
<td class="right">idx</td>
<td class="right">x1</td>
<td class="right">x2</td>
<td class="right">x3</td>
</tr>

<tr>
<td class="right">1</td>
<td class="right">0.3668076837657581</td>
<td class="right">0.6132177634585174</td>
<td class="right">0.7271614060006228</td>
</tr>

<tr>
<td class="right">2</td>
<td class="right">0.8402870142445031</td>
<td class="right">0.8268085074961338</td>
<td class="right">0.7631093687543671</td>
</tr>

<tr>
<td class="right">3</td>
<td class="right">0.6880695323767783</td>
<td class="right">0.03210864954311465</td>
<td class="right">0.9955387409975662</td>
</tr>

<tr>
<td class="right">4</td>
<td class="right">0.8182674804881986</td>
<td class="right">0.7627315792913878</td>
<td class="right">0.966201684866371</td>
</tr>
</tbody>
</table>
</div>
</div>

<div id="outline-container-sec-4" class="outline-2">
<h2 id="sec-4"><span class="section-number-2">4</span> Indexing</h2>
<div class="outline-text-2" id="text-4">
<p>
The idea of <code>getindex</code> is to stick with the behavior of <code>DataFrames</code>
as far as possible for the basics, while extending it to allow
indexing of rows by dates. Hence, indexing <code>TimeData</code> types should
hopefully fit seamlessly into behavior familiar from other important
types, with only intuitive extensions. However, it is important to
note that indexing deviates from <code>DataFrame</code> behavior in one aspect:
<code>getindex</code> will NEVER change the type of the variable! If you call it
on a <code>Timematr</code> variable, it will also return a <code>Timematr</code> variable,
and if you call it on type <code>Timenum</code> it will return <code>Timenum</code> as well.
This behavior does deviate from <code>DataFrame</code> behavior in such that, for
example, <code>DataFrames</code> return <code>Array</code> for single columns.
</p>

<div class="org-src-container">

<pre class="src src-julia">typeof(valsDf[:, 1])
typeof(td[:, 1])

typeof(valsDf[1, 1])
typeof(td[1, 1])

## empty instance
typeof(td[4:3, 5:4])
</pre>
</div>
<p>
This will print:
</p>
<pre class="example">
Array{Float64,1}
Timedata{Int64} (constructor with 1 method)

Float64
Timedata{Int64} (constructor with 1 method)


Timedata{Int64} (constructor with 1 method)
</pre>

<p>
Possible ways of indexing are:
</p>
<div class="org-src-container">

<pre class="src src-julia">## indexing by numeric indices
tmp = tm[2:3]
tmp = tm[1:3, 1:2]
tmp = tm[2, :]
tmp = tm[2]
tmp = tm[1:2, 2]
tmp = tm[3, 3]

## indexing with column names as symbols
tmp = tm[:A]
tmp = tm[2, [:A, :B]]

## logical indexing
logicCol = [true, false, true]
logicRow = repmat([true, false], 2, 1)[:]
tmp = tm[logicCol]
tmp = tm[logicRow, logicCol]
tmp = tm[logicRow, :]

## indexing with Dates
DatesToFind = [Date(2013, 7, ii) for ii=2:3]
tmp = tm[DatesToFind]
tm[Date(2013,7,1):Date(2013,7,3)]
tm[Date(2013,7,2):Date(2013,7,3), :B]
tm[Date(2013,7,3):Date(2013,7,12), [true, false, false]]
</pre>
</div>

<p>
As all <code>getindex</code> methods are designed to retain the <code>TimeData</code>
structure, we need an additional <code>get</code> method to actual get the value
of an entry itself.
</p>

<div class="org-src-container">

<pre class="src src-julia">## returning the first value only
get(tm, 1, 1)

## returning all values as Array{Any,2}
kk = get(tm)
isa(kk, Array{Any})
</pre>
</div>

<pre class="example">
true
</pre>
</div>
</div>

<div id="outline-container-sec-5" class="outline-2">
<h2 id="sec-5"><span class="section-number-2">5</span> Read, write, io</h2>
<div class="outline-text-2" id="text-5">
<p>
Data can easily be imported from csv-files using function
<code>readTimedata</code>. Under the hood, the function makes use of <code>readtable</code>
from the <code>DataFrames</code> package. Additionally, columns are parsed for
dates. The first column matching the regexp for dates will be chosen
as time identifier.
</p>
<div class="org-src-container">

<pre class="src src-julia">filePath = joinpath(Pkg.dir("TimeData"), "data", "logRet.csv");
tm = readTimedata(filePath)
tm[1:5, 1:4]
</pre>
</div>

<table border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides">


<colgroup>
<col  class="right" />

<col  class="right" />

<col  class="right" />

<col  class="right" />

<col  class="right" />
</colgroup>
<tbody>
<tr>
<td class="right">idx</td>
<td class="right">MMM</td>
<td class="right">ABT</td>
<td class="right">ACE</td>
<td class="right">ACT</td>
</tr>

<tr>
<td class="right">2012-01-03</td>
<td class="right">2.12505</td>
<td class="right">0.88718</td>
<td class="right">0.29744</td>
<td class="right">0.47946</td>
</tr>

<tr>
<td class="right">2012-01-04</td>
<td class="right">0.82264</td>
<td class="right">-0.38476</td>
<td class="right">-0.95495</td>
<td class="right">-0.52919</td>
</tr>

<tr>
<td class="right">2012-01-05</td>
<td class="right">-0.44787</td>
<td class="right">-0.23157</td>
<td class="right">0.28445</td>
<td class="right">2.74752</td>
</tr>

<tr>
<td class="right">2012-01-06</td>
<td class="right">-0.51253</td>
<td class="right">-0.93168</td>
<td class="right">0.23891</td>
<td class="right">1.94894</td>
</tr>

<tr>
<td class="right">2012-01-09</td>
<td class="right">0.58732</td>
<td class="right">0</td>
<td class="right">0.46128</td>
<td class="right">0.28436</td>
</tr>
</tbody>
</table>

<p>
In the REPL itself, Julia calls the <code>display</code> method to show
information about an instance.
</p>

<div class="org-src-container">

<pre class="src src-julia">tm
</pre>
</div>

<pre class="example">
type: Timematr{Date}
dimensions: (333,348)
333x6 DataFrame
|-------|------------|----------|----------|----------|----------|----------|
| Row # | idx        | MMM      | ABT      | ACE      | ACT      | ADBE     |
| 1     | 2012-01-03 | 2.12505  | 0.88718  | 0.29744  | 0.47946  | 1.0556   |
| 2     | 2012-01-04 | 0.82264  | -0.38476 | -0.95495 | -0.52919 | -1.02024 |
| 3     | 2012-01-05 | -0.44787 | -0.23157 | 0.28445  | 2.74752  | 0.70472  |
| 4     | 2012-01-06 | -0.51253 | -0.93168 | 0.23891  | 1.94894  | 0.83917  |
| 5     | 2012-01-09 | 0.58732  | 0.0      | 0.46128  | 0.28436  | -0.66376 |
| 6     | 2012-01-10 | 0.52193  | 0.46693  | 1.31261  | 1.85986  | 2.32125  |
| 7     | 2012-01-11 | -0.63413 | -0.38895 | -1.52066 | -3.06604 | 0.41012  |
| 8     | 2012-01-12 | 0.60934  | -0.46875 | 0.50453  | -0.93039 | -0.30743 |
⋮
| 325   | 2013-04-19 | 0.69118  | 0.86745  | 0.77089  | 1.84469  | 0.6278   |
| 326   | 2013-04-22 | 0.08606  | -0.84023 | 0.27067  | -0.64178 | -0.47048 |
| 327   | 2013-04-23 | 1.48952  | 0.86721  | 0.8188   | 0.93582  | 0.76063  |
| 328   | 2013-04-24 | 0.451    | -1.8794  | -0.51518 | -0.49734 | -0.44673 |
| 329   | 2013-04-25 | -2.81414 | -0.08252 | -0.04492 | 0.61876  | 0.84708  |
| 330   | 2013-04-26 | -1.04683 | -0.08259 | -0.63106 | 2.05182  | -0.31125 |
| 331   | 2013-04-29 | 0.03897  | 0.74085  | -0.02261 | 4.49427  | 0.33344  |
| 332   | 2013-04-30 | 0.84381  | 0.51807  | 0.24845  | 0.14197  | 0.04438  |
| 333   | 2013-05-01 | -0.14498 | -0.08162 | -0.94057 | -1.27548 | -0.82415 |
</pre>


<p>
As one can see, the <code>display</code> method will show the type of the
variable, together with its dimensions and a snippet of the first
values. Note that the number of columns does not entail the dates
column, but does only count the columns of the remaining variables.
Inherently, <code>display</code> makes use of the display method that is
implemented for <code>DataFrames</code>, which is the reason for the somewhat
misleading output line <code>333x348 DataFrame:</code>. An issue that still needs
to be fixed. However, html display in IJulia already shows an improved
table output.
</p>

<p>
An even more elaborate way of looking at the data contained in a
<code>TimeData</code> type is function <code>str</code> (following the name used in R),
which will print:
</p>

<div class="org-src-container">

<pre class="src src-julia">## str(tm) # uncomment for execution
</pre>
</div>

<pre class="example">
type: Timematr{Date}
:vals  		  DataFrame
:idx  		  Array{Date,1}

dimensions of vals: (333,348)

-------------------------------------------
From: 2012-01-03, To: 2013-05-01
-------------------------------------------

333x6 DataFrame
|-------|------------|----------|----------|----------|----------|----------|
| Row # | idx        | MMM      | ABT      | ACE      | ACT      | ADBE     |
| 1     | 2012-01-03 | 2.12505  | 0.88718  | 0.29744  | 0.47946  | 1.0556   |
| 2     | 2012-01-04 | 0.82264  | -0.38476 | -0.95495 | -0.52919 | -1.02024 |
| 3     | 2012-01-05 | -0.44787 | -0.23157 | 0.28445  | 2.74752  | 0.70472  |
| 4     | 2012-01-06 | -0.51253 | -0.93168 | 0.23891  | 1.94894  | 0.83917  |
| 5     | 2012-01-09 | 0.58732  | 0.0      | 0.46128  | 0.28436  | -0.66376 |
| 6     | 2012-01-10 | 0.52193  | 0.46693  | 1.31261  | 1.85986  | 2.32125  |
| 7     | 2012-01-11 | -0.63413 | -0.38895 | -1.52066 | -3.06604 | 0.41012  |
| 8     | 2012-01-12 | 0.60934  | -0.46875 | 0.50453  | -0.93039 | -0.30743 |
⋮
| 325   | 2013-04-19 | 0.69118  | 0.86745  | 0.77089  | 1.84469  | 0.6278   |
| 326   | 2013-04-22 | 0.08606  | -0.84023 | 0.27067  | -0.64178 | -0.47048 |
| 327   | 2013-04-23 | 1.48952  | 0.86721  | 0.8188   | 0.93582  | 0.76063  |
| 328   | 2013-04-24 | 0.451    | -1.8794  | -0.51518 | -0.49734 | -0.44673 |
| 329   | 2013-04-25 | -2.81414 | -0.08252 | -0.04492 | 0.61876  | 0.84708  |
| 330   | 2013-04-26 | -1.04683 | -0.08259 | -0.63106 | 2.05182  | -0.31125 |
| 331   | 2013-04-29 | 0.03897  | 0.74085  | -0.02261 | 4.49427  | 0.33344  |
| 332   | 2013-04-30 | 0.84381  | 0.51807  | 0.24845  | 0.14197  | 0.04438  |
| 333   | 2013-05-01 | -0.14498 | -0.08162 | -0.94057 | -1.27548 | -0.82415 |
</pre>

<p>
This additionally shows the names of the fields of the object, and
also explicitly displays the time period of the data.
</p>

<p>
To save an object to disk, simply call function <code>writeTimedata</code>, which
internally uses <code>writetable</code> from the <code>DataFrame</code> package. In
accordance with <code>writetable</code>, the first argument is the filename as
string, while the second argument is the variable to be saved.
</p>

<div class="org-src-container">

<pre class="src src-julia">#   writeTimedata("data/logRet2.csv", tm) # uncomment for execution
</pre>
</div>
</div>
</div>

<div id="outline-container-sec-6" class="outline-2">
<h2 id="sec-6"><span class="section-number-2">6</span> Conversion</h2>
<div class="outline-text-2" id="text-6">
<p>
To DataFrame:
</p>
<ul class="org-ul">
<li>td.vals (without index)
</li>
<li>convert(DataFrame, td)
</li>
</ul>

<p>
Get column as DataArray:
</p>
<ul class="org-ul">
<li>td.vals[ii]
</li>
<li>td.vals[:, ii]
</li>
</ul>

<p>
As array:
</p>
<ul class="org-ul">
<li>asArr
</li>
</ul>

<p>
Array with NaN to DataFrame with NA:
</p>
<ul class="org-ul">
<li>still missing?
</li>
</ul>

<p>
Remove NAs:
</p>
<ul class="org-ul">
<li>dropna (DataArray)
</li>
<li>complete<sub>cases</sub> (DataFrame)
</li>
<li>narm (TimeData)
</li>
</ul>
</div>
</div>

<div id="outline-container-sec-7" class="outline-2">
<h2 id="sec-7"><span class="section-number-2">7</span> Data manipulation</h2>
<div class="outline-text-2" id="text-7">
<p>
The following data manipulation functions could be used from
<code>DataFrames</code> and are not yet implemented for <code>TimeData</code> types yet. 
</p>

<ul class="org-ul">
<li>sort
</li>
<li>select / groupby
</li>
<li>stack / melt (wide to long format)
</li>
<li>unstack / cast (long to wide)
</li>
<li>merge / join
</li>
<li>normalization
</li>
</ul>
</div>
</div>

<div id="outline-container-sec-8" class="outline-2">
<h2 id="sec-8"><span class="section-number-2">8</span> Functions and operators</h2>
<div class="outline-text-2" id="text-8">
<p>
Mathematical operators and functions are only implemented for
<code>Timematr</code> and <code>Timenum</code> types, since they are not well defined
operations for general data (strings, &#x2026;).
</p>

<p>
Whenever possible, functions apply element-wise to observations only,
and you should get back the same type that you did call the function
on. In case that this is not possible, the type that you get back
should be the natural first choice. For example, element-wise
comparisons should return a logical value for each entry, which by
definition could not be of type <code>Timenum</code> where only numeric values
are allowed.
</p>

<div class="org-src-container">

<pre class="src src-julia">typeof(tm .+ tm)
typeof(tm .&gt; 0.5)
</pre>
</div>

<pre class="example">
Timematr{Date} (constructor with 1 method)
Timedata{Date} (constructor with 1 method)
</pre>

<p>
The standard library for <code>TimeData</code> comprises all standard operators
and mathematical functions. As expected, these functions all apply
elementwise, and leave the time information untouched. Where
additional arguments are allowed for <code>Arrays</code>, they are allowed for
<code>TimeData</code> types as well.
</p>

<div class="org-src-container">

<pre class="src src-julia">tm[1:3, 1:3] .&gt; 0.5
exp(tm[1:3, 1:3])
round(tm[1:3, 1:3], 2)
</pre>
</div>

<pre class="example">
type: Timedata{Date}
dimensions: (3,3)
3x4 DataFrame
|-------|------------|-------|-------|-------|
| Row # | idx        | MMM   | ABT   | ACE   |
| 1     | 2012-01-03 | true  | true  | false |
| 2     | 2012-01-04 | true  | false | false |
| 3     | 2012-01-05 | false | false | false |

type: Timematr{Date}
dimensions: (3,3)
3x4 DataFrame
|-------|------------|----------|----------|----------|
| Row # | idx        | MMM      | ABT      | ACE      |
| 1     | 2012-01-03 | 8.37332  | 2.42827  | 1.34641  |
| 2     | 2012-01-04 | 2.2765   | 0.680614 | 0.384831 |
| 3     | 2012-01-05 | 0.638988 | 0.793287 | 1.32903  |

type: Timematr{Date}
dimensions: (3,3)
3x4 DataFrame
|-------|------------|-------|-------|-------|
| Row # | idx        | MMM   | ABT   | ACE   |
| 1     | 2012-01-03 | 2.13  | 0.89  | 0.3   |
| 2     | 2012-01-04 | 0.82  | -0.38 | -0.95 |
| 3     | 2012-01-05 | -0.45 | -0.23 | 0.28  |
</pre>
</div>
</div>

<div id="outline-container-sec-9" class="outline-2">
<h2 id="sec-9"><span class="section-number-2">9</span> Iterators and map</h2>
<div class="outline-text-2" id="text-9">
<p>
The idea of iterators and <code>map</code> is to allow application of a given
function successively to parts of the data set, such that the
resulting values will automatically be embedded into the metadata
again. Applying a function iteratively and simply capturing the
results in an <code>Array{Any, 1}</code> is a quite easy task with
comprehensions. However, automatically storing the data in a
meaningful way encapsulated by its respective metadata is the hard
part of the task. This, however, can be easily achieved through a
combination of iterators and <code>map</code> for a quite general set tasks.
</p>
</div>

<div id="outline-container-sec-9-1" class="outline-3">
<h3 id="sec-9-1"><span class="section-number-3">9.1</span> Iterators</h3>
<div class="outline-text-3" id="text-9-1">
<p>
For rectangular data, there basically exist three ways of iteratively
stepping through:
</p>
<ul class="org-ul">
<li>element-wise
</li>
<li>row-wise
</li>
<li>column-wise
</li>
</ul>

<p>
In addition, functions can either apply to individual values only or
to values that are embedded with metadata (values plus time
information and / or variable name). Hence, the package implements six
different standard iterators, each returning different slices of a
given <code>TimeData</code> object:
</p>

<table border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides">


<colgroup>
<col  class="left" />

<col  class="left" />

<col  class="left" />
</colgroup>
<thead>
<tr>
<th scope="col" class="left">&#xa0;</th>
<th scope="col" class="left">value</th>
<th scope="col" class="left">value plus metadata</th>
</tr>
</thead>
<tbody>
<tr>
<td class="left">element-wise</td>
<td class="left">eachentry</td>
<td class="left">eachobs</td>
</tr>

<tr>
<td class="left">row-wise</td>
<td class="left">eachrow</td>
<td class="left">eachdate</td>
</tr>

<tr>
<td class="left">column-wise</td>
<td class="left">eachcol</td>
<td class="left">eachvar</td>
</tr>
</tbody>
</table>

<p>
All iterators containing metadata will provide slices of the data in
the exact same type as the original data. Hence, <code>eachdate</code> applied to
a <code>Timematr</code> will return <code>Timematr</code> objects of dimension \(1\times m\),
while <code>eachvar</code> applied to a <code>Timedata</code> object will return \(n \times
1\) <code>Timedata</code> objects.
</p>

<p>
Iterators without metadata provide data in the following formats
</p>
<ul class="org-ul">
<li><code>eachentry</code>: individual entries only
</li>
<li><code>eachrow</code>: \(1\times m\) <code>DataFrame</code> - this is different to <code>eachrow</code>
  for <code>DataFrames</code>
</li>
<li><code>eachcol</code>: tuple (nam, col), where nam is the variable name as
<code>Symbol</code> and col is the data as <code>DataArray</code> or <code>Array</code>
</li>
</ul>
</div>
</div>

<div id="outline-container-sec-9-2" class="outline-3">
<h3 id="sec-9-2"><span class="section-number-3">9.2</span> Map</h3>
<div class="outline-text-3" id="text-9-2">
<p>
With iterators specifying how to slice a given data set, we still need
to define what type of function should be applied to each part.
Thereby, we again distinguish functions into two groups. Functions
that preserve the original dimensions of the data are implemented
through function <code>map</code>, while functions that reduce the original
dimension are implemented through <code>collapse</code> (see below).
</p>

<p>
Hence, any operation that will map a data row to a row again, and a
data column to a column again, is implemented through function <code>map</code>.
Furthermore, as we do not know a priori which type to expect of an
arbitrary function, output values will always be captured in the most
general format as type <code>Timedata</code>. Hence, the output of any function
applied iteratively to data with <code>map</code> will always be a <code>Timedata</code>
object of dimensions equal to the original data set.
</p>

<p>
Although we know that <code>map</code> will always map a row to a row, there
still are multiple formats that conceptually could represent row-wise
data: an \(1 \times m\) <code>DataFrame</code>, an \(1 \times m\) <code>TimeData</code> object,
an \(1 \times m\) <code>Array</code> or an \(m \times 1\) <code>Array</code>. In other words:
some mapping functions preserve metadata, and some don't.
</p>

<p>
So far, <code>map</code> automatically handles several function output formats
for any given input iterator:
</p>

<table border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides">


<colgroup>
<col  class="left" />

<col  class="left" />
</colgroup>
<thead>
<tr>
<th scope="col" class="left">iterator</th>
<th scope="col" class="left">output formats</th>
</tr>
</thead>
<tbody>
<tr>
<td class="left"><code>eachentry</code></td>
<td class="left">a single value of any standard type</td>
</tr>

<tr>
<td class="left"><code>eachrow</code></td>
<td class="left">mx1 <code>Array</code> or 1xm <code>Array</code>, 1xm <code>DataFrame</code></td>
</tr>

<tr>
<td class="left"><code>eachcol</code></td>
<td class="left">nx1 <code>Array</code> or <code>DataArray</code></td>
</tr>

<tr>
<td class="left"><code>eachobs</code></td>
<td class="left">1x1 <code>Array</code>, <code>DataFrame</code> or <code>TimeData</code></td>
</tr>

<tr>
<td class="left"><code>eachdate</code></td>
<td class="left">mx1 <code>Array</code> or 1xm <code>Array</code>, 1xm <code>DataFrame</code> or <code>TimeData</code></td>
</tr>

<tr>
<td class="left"><code>eachvar</code></td>
<td class="left">nx1 <code>Array</code> or <code>DataFrame</code> or <code>DataArray</code> or <code>TimeData</code></td>
</tr>
</tbody>
</table>

<p>
Internally, column data is extracted from possible metadata by
function <code>getColData</code>, row data by <code>nthRowElem</code> and singleton entries
by <code>get(x, :)</code>. Only the extracted values themselves then will be used
in the output.
</p>
</div>
</div>

<div id="outline-container-sec-9-3" class="outline-3">
<h3 id="sec-9-3"><span class="section-number-3">9.3</span> Collapse</h3>
<div class="outline-text-3" id="text-9-3">
<p>
While <code>map</code> is designed to always match the dimension of the original
input data, there are also situations where a function collapses a
complete row or column to a single value. Such mappings are
implemented by two collapse functions, each of them being accessible
with two different iterators: one returning values only, and one
returning values with metadata. Also, the output of both functions
differs:
</p>

<ul class="org-ul">
<li><code>collapseDates</code>: 
<ul class="org-ul">
<li>output: \(1 \times m\) <code>DataFrame</code>
</li>
<li>possible iterators: <code>eachrow</code>, <code>eachdate</code>
</li>
</ul>
</li>
<li><code>collapseVars</code>: 
<ul class="org-ul">
<li>output: \(n \times 1\) <code>TimeData</code>
</li>
<li>possible iterators: <code>eachcol</code>, <code>eachvar</code>
</li>
</ul>
</li>
</ul>
</div>
</div>

<div id="outline-container-sec-9-4" class="outline-3">
<h3 id="sec-9-4"><span class="section-number-3">9.4</span> Check condition</h3>
<div class="outline-text-3" id="text-9-4">
<p>
For some special cases there exist more fine-tailored implementations
of iterative data mapping in order to allow for more efficient data
handling. For example, <code>chk...</code> functions are successively checking
whether a given condition is fulfilled for the individual data slices,
and hence will return one of only three possible values for each
slice:
</p>
<ul class="org-ul">
<li><code>true</code>
</li>
<li><code>false</code>
</li>
<li><code>NA</code>
</li>
</ul>

<p>
Again, we can check whether a certain condition holds based on three
different ways of slicing the data: the condition can be a property of
either individual values (<code>chkElw</code>), individual rows (<code>chkDates</code>) or
individual columns (<code>chkVars</code>). This way, output dimensions and
formats will vary depending on the condition used:
</p>
<ul class="org-ul">
<li><code>chkElw</code>: \(n \times m\) <code>TimeData</code> object
</li>
<li><code>chkDates</code>: \(n \times 1\) <code>TimeData</code> object
</li>
<li><code>chkVars</code>: \(1 \times m\) <code>DataFrame</code>
</li>
</ul>
<p>
Still, however, all individual entries of any output must be either
<code>Bool</code> or <code>NA</code>.
</p>

<p>
Also, a certain condition can be a property of the plain data values
themselves, or of data combined with metadata. Hence, all three
<code>chk...</code> functions again allow for two different iterators each:
</p>
<ul class="org-ul">
<li><code>chkElw</code>: either <code>eachentry</code> or <code>eachobs</code>
</li>
<li><code>chkDates</code>: either <code>eachrow</code> or <code>eachdate</code>
</li>
<li><code>chkVars</code>: either <code>eachcol</code> or <code>eachvar</code>
</li>
</ul>

<p>
One frequent application of <code>chk...</code> functions is selecting data
slices based on whether a certain condition is fulfilled. This
selection generally comprises three steps:
</p>
<ul class="org-ul">
<li>checking whether the condition is met for data slices
</li>
<li>deciding how to interpret slices with unknown property regarding the
condition: should <code>NA</code> be interpreted as <code>true</code> or <code>false</code>?
</li>
<li>converting the result to <code>Array{Bool, 1}</code> or <code>Array{Bool, 2}</code> to use
it for logical indexing
</li>
</ul>

<p>
Thereby the selection of certain rows or columns will again result in
a rectangular shaped data set. <code>chkElw</code>, however, selects individual
entries from possibly different columns and rows, so that the data has
to be returned as <code>TimeData</code> object in long format: in addition to the
index, the result comprises columns <code>variable</code> and <code>value</code>.
</p>

<p>
For example, selecting all columns with a maximum value above 10:
</p>
<div class="org-src-container">

<pre class="src src-julia">filePath = joinpath(Pkg.dir("TimeData"), "data", "logRet.csv");
tm = readTimedata(filePath)


columnsMeetingCondition = chkVars(x -&gt; minimum(x, 1)[1, 1] .&lt; -25, eachvar(tm)) |&gt;
                          x -&gt; asArr(x, Bool, false) |&gt;
                          x -&gt; tm[x[:]]

minimum(columnsMeetingCondition, 1)
</pre>
</div>

<table border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides">


<colgroup>
<col  class="right" />

<col  class="right" />

<col  class="right" />
</colgroup>
<tbody>
<tr>
<td class="right">APOL</td>
<td class="right">CTL</td>
<td class="right">FOSL</td>
</tr>

<tr>
<td class="right">-25.04314</td>
<td class="right">-25.61743</td>
<td class="right">-47.11015</td>
</tr>
</tbody>
</table>
</div>
</div>
</div>

<div id="outline-container-sec-10" class="outline-2">
<h2 id="sec-10"><span class="section-number-2">10</span> Plotting</h2>
<div class="outline-text-2" id="text-10">
<p>
problem with dates: 
</p>
<ul class="org-ul">
<li>as strings:
<ul class="org-ul">
<li>not working for Winston
</li>
<li>interpreted as categorical data by gadfly: very slow
</li>
</ul>
</li>
</ul>
</div>

<div id="outline-container-sec-10-1" class="outline-3">
<h3 id="sec-10-1"><span class="section-number-3">10.1</span> Winston</h3>
</div>

<div id="outline-container-sec-10-2" class="outline-3">
<h3 id="sec-10-2"><span class="section-number-3">10.2</span> Gadfly</h3>
<div class="outline-text-3" id="text-10-2">
<p>
only numeric values
</p>
</div>
</div>
</div>

<div id="outline-container-sec-11" class="outline-2">
<h2 id="sec-11"><span class="section-number-2">11</span> Additional functions</h2>
<div class="outline-text-2" id="text-11">
<p>
Besides basic mathematical functions and operators, there are already
quite some additional functions that are defined for several
<code>TimeData</code> types. You can find them in the <a href="http://juliafinmetrix.readthedocs.org/en/latest/timeData.html?highlight=setindex!">online documentation</a>.
</p>
</div>
</div>

<div id="outline-container-sec-12" class="outline-2">
<h2 id="sec-12"><span class="section-number-2">12</span> Under the hood: implementation</h2>
<div class="outline-text-2" id="text-12">
<p>
The balancing act between emulating and extending <code>DataFrames</code> is
implemented in Julia maybe a bit less naturally than in traditional
object oriented programming languages. There, one can easily inherit
behavior from other classes through subclasses, thereby overwriting
inherited methods whenever desired. In Julia, however, composite types
are not allowed to be subtypes of other composite types, but only
abstract types may act as parent. Under the hood, <code>TimeData</code> types
hence inherit their behavior by owning a field of type <code>DataFrame</code>.
This way, functions can easily be delegated to this field whenever
necessary. For a more elaborate discussion on this topic and the
interior design of <code>TimeData</code>, take a look at <a href="http://grollchristian.wordpress.com/2014/01/22/julia-inheriting-behavior/">this post</a> on my blog. 
</p>
</div>
</div>

<div id="outline-container-sec-13" class="outline-2">
<h2 id="sec-13"><span class="section-number-2">13</span> Current state</h2>
<div class="outline-text-2" id="text-13">
<p>
All <code>TimeData</code> types should already provide a convenient way to
represent and handle time series data. In extension to the already
implemented functions directly applying to types of the <code>TimeData</code>
package, any <code>DataFrame</code> functionality in principle can easily be
regained by delegating functions to field <code>vals</code>. So far, I only
tested <code>TimeData</code> types with <code>Date</code> type from the <code>Dates</code> package
myself, and not yet with type <code>DateTime</code>.
</p>
</div>
</div>


<div id="outline-container-sec-14" class="outline-2">
<h2 id="sec-14"><span class="section-number-2">14</span> Acknowledgement</h2>
<div class="outline-text-2" id="text-14">
<p>
Of course, any package can only be as good as the individual parts
that it builds on. Accordingly, I'd like to thank all people that
were involved in the development of all the functions that were made
ready to use for me to build this package upon. In particular, I want
to thank the developers of
</p>
<ul class="org-ul">
<li>the <b>Julia language</b>, for their continuous and tremendous efforts
during the creation of this free, fast and highly flexible
programming language!
</li>
<li>the <b>DataFrames</b> package, which definitely provides the best
representation for general types of data in data analysis. It's a
role model that every last bit of code of <code>TimeData</code> depends on, and
the interface that every statistics package should use.
</li>
<li>the <b>Dates</b> package, which is a thoughtful implementation of
dates, time and durations, and the backbone of all time components
in <code>TimeData</code>.
</li>
<li>the <b>TimeSeries</b> package, which follows a different approach to
handling time series data. Having a quite similar goal in mind, the
package was a great inspiration for me, and occasionally I even
could borrow parts of code from it (for example, from an old version
of function <code>readtime</code>).
</li>
</ul>
</div>
</div>
</div>
<div id="postamble" class="status">
<p class="creator"><a href="http://www.gnu.org/software/emacs/">Emacs</a> 24.3.1 (<a href="http://orgmode.org">Org</a> mode 8.2.7)</p>
<p class="validation"><a href="http://validator.w3.org/check?uri=referer">Validate</a></p>
</div>
</body>
</html>
